Improving implementation of linear discriminant analysis for the high dimension/small sample size problem

Classification based on Fisher’s Linear Discriminant Analysis (FLDA) is challenging when the number of variables largely exceeds the number of given samples: The original FLDA needs to be carefully modified and with high dimensionality implementational issues like reduction of storage costs are of crucial importance. In this paper we first review popular methods for the high dimension/small sample size problem and choose the one closest, in some sense, to the classical regular approach. Then we improve its implementation with regards to computational and storage costs and numerical stability. This is achieved through combining a variety of known and new implementation strategies. The paper demonstrates in experiments the superiority, with respect to both overall costs and classification rates, of the resulting algorithm compared with other methods.